In computer vision applications, a wide variety of geometrical problems are solved using image features, such as points and lines. Conventional methods use all the available features and solve the problem using least squares measure over all of the features. However, in many computer vision problems, minimal solutions have proven to be less noise-prone compared to non-minimal methods.
Minimal solutions have been very useful as hypothesis generators in hypothesize-and-test procedures such as RANdom SAmple Consensus (RANSAC), see U.S. Pat. No. 7,359,526.
Minimal solutions are known for several computer vision problems: auto-calibration of radial distortion, perspective three point problem, the five point relative pose problem, the six point focal length problem, the six point generalized camera problem, the nine point problem for estimating para-catadioptric fundamental matrices, and the nine point radial distortion problem. There are also unification efforts to keep track of all the known solutions.
2D-3D Registration
The invention is particularly concerned with pose estimation using points and lines. A pose is defined as having three degrees of freedom in translation and rotation. Given three correspondences between points and lines in a world reference frame, and their projections on an image in a camera reference frame, the goal is determine the pose between the camera in the world reference frames. Solution for three lines, or three points configurations are known. As defined herein, a “reference frame” is a 3D coordinate system.
In practice, both point and line features have complementary advantages. In deference to our intended application of geo-localization in urban environments, using image sequences and coarse 3D model, it is relatively easy to obtain the correspondences between lines in the 3D model and the images.
Point matching between two images is also well-known, and easier than line matching. Although, the fusion of points and lines for tracking has been described, a minimal solution, which is less sensitive to outliers, insufficient correspondences and narrow field of view, have not been described in the prior art.
Recently, minimal pose procedures for stereo cameras have been described using either points or lines, but not both.
Image-Based Geo-Localization
There has been an increasing interest in inferring geo-locations from images. It is possible to obtain a geo-spatial localization by matching a query image with an image stored in database using a vanishing vertical direction.
Several 3D reconstruction procedures have been described for the reconstruction of large urban environments. One localization method matches lines from a 3D model with lines in images acquired of a controlled indoor environment by a wide angle camera.
However, determining poses in outdoor environments, such as “urban canyons,” is a more difficult challenge. It is well known that pose estimation for the purpose of geo-location in such environments is often difficult with GPS-based solutions where radio reception is poor.
Therefore, it is desired to provide camera-based solution for such geo-location applications.